Based on Actual Images

Demand is growing for more realistic three-dimensional (3-D) image spaces that can enhance interpersonal communication. Existing 3-D image space systems are mainly based on computer graphics (CG); examples include Cybertown [1] and Interspace [2]. Unfortunately, the image spaces created by existing CG techniques are unnatural and certainly not attractive. There are two problems in creating photo-realistic 3-D image spaces. First, how can we simplify the modeling/rendering process to reproduce actual scenes and objects? In the research field of computer vision (CV), one key point is how the scenes and objects are to be represented in the computer. Second, how can we improve image quality while minimizing the computing power needed? Texture mapping, commonly used in CG techniques, is useful for creating photo-realistic 3-D image spaces, but it requires too much handwork and large computing power is needed for constructing and walking through the resulting image spaces. The limitations imposed by modeling/rendering budgets, computing power, and telecommunication network speeds means that such spaces are unrealistic, so a new paradigm is needed. QuickTimeVR was one of first techniques to suggest that the traditional modeling/rendering process could be skipped [3]. The major limitation of this technique is that the viewpoint is fixed. Devec suggested the approach of combining modeling with rendering, i.e., combining geometry-based and image-based techniques [4]. This approach requires only a sparse set of images and can produce photo-realistic images from arbitrary viewpoints. Seitz discussed image-based rendering with the goal of allowing a user to receive correctly modeled view transforms [5]. Image-based rendering techniques using the plenoptic function were proposed for rendering images of the environment from any arbitrary viewpoint. Cylindrical projection was proposed in McMillan’s plenoptic modeling as the plenoptic function [6]. Both Levoy’s light field rendering [7] and Gortler’s lumigraph [8] use the light slab method and attempt to sample fully the plenoptic function using parameter subsets. In the former, many images are captured by a com-